<html>
<head><meta charset="utf-8"><title>metadata size · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html">metadata size</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="242523343"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523343" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523343">(Jun 13 2021 at 18:12)</a>:</h4>
<p>I was bisecting recently and I noticed that the downloaded <code>std</code> was 20 MB, which is five times the size of the cargo binary, which is also downloaded. I'm curious if anyone knows what parts of metadata are taking up the most space?</p>



<a name="242523487"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523487" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523487">(Jun 13 2021 at 18:16)</a>:</h4>
<p>The metadata for libstd.rlib is 6.5MB out of 16MB for me. Note that when linking executables not just the crate metadata is discarded, but also all trivially unused functions and statics thanks to <code>--gc-sections</code>.</p>



<a name="242523512"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523512" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523512">(Jun 13 2021 at 18:17)</a>:</h4>
<p>To get a breakdown of metadata usage you can use <code>-Zmeta-stats</code> while compiling the sysroot.</p>



<a name="242523653"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523653" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523653">(Jun 13 2021 at 18:20)</a>:</h4>
<p>This is what I get:</p>
<div class="codehilite"><pre><span></span><code>metadata stats:
             dep bytes: 725
     lib feature bytes: 7172
       lang item bytes: 30
 diagnostic item bytes: 152
          native bytes: 0
      source_map bytes: 93341
            impl bytes: 11570
    exp. symbols bytes: 5566
  def-path table bytes: 365030
 proc-macro-data-bytes: 0
             mir bytes: 2510317
            item bytes: 2695978
           table bytes: 1830444
         hygiene bytes: 3675259
            zero bytes: 4088925
           total bytes: 11196286
</code></pre></div>



<a name="242523673"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523673" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523673">(Jun 13 2021 at 18:21)</a>:</h4>
<p>Wow, that is a <em>lot</em> of zero bytes</p>



<a name="242523676"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523676" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523676">(Jun 13 2021 at 18:21)</a>:</h4>
<p>What are they for?</p>



<a name="242523679"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523679" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523679">(Jun 13 2021 at 18:21)</a>:</h4>
<blockquote>
<p>The metadata for libstd.rlib is 6.5MB out of 16MB for me.</p>
</blockquote>
<p>Ah, yeah, I forgot that object code is embedded as well.</p>



<a name="242523718"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523718" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523718">(Jun 13 2021 at 18:22)</a>:</h4>
<p>By the way, why is MIR embedded in addition to object code?</p>



<a name="242523737"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523737" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523737">(Jun 13 2021 at 18:22)</a>:</h4>
<p><span class="user-mention" data-user-id="307537">@Noah Lev</span> for generic functions, they are monomorphized in downstream crates</p>



<a name="242523738"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523738" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523738">(Jun 13 2021 at 18:22)</a>:</h4>
<p>Are there docs somewhere on what each of these keys mean, or should I just grep through teh source?</p>



<a name="242523740"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523740" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523740">(Jun 13 2021 at 18:22)</a>:</h4>
<p>MIR is embeded for <code>#[inline]</code> and generic functions.</p>



<a name="242523747"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523747" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523747">(Jun 13 2021 at 18:23)</a>:</h4>
<p>This is all printed in <code>compiler/rustc_metadata/src/rmeta/encoder.rs</code></p>



<a name="242523750"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523750" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523750">(Jun 13 2021 at 18:23)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523737">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="307537">Noah Lev</span> for generic functions, they are monomorphized in downstream crates</p>
</blockquote>
<p>Oh, of course! So do monomorphic functions just have object code and polymorphic functions just have MIR, or do monomorphic functions have MIR too?</p>



<a name="242523751"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523751" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523751">(Jun 13 2021 at 18:23)</a>:</h4>
<p>I don't think there are any docs for it.</p>



<a name="242523754"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523754" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523754">(Jun 13 2021 at 18:23)</a>:</h4>
<p><span class="user-mention silent" data-user-id="133247">bjorn3</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523740">said</a>:</p>
<blockquote>
<p>MIR is embeded for <code>#[inline]</code> and generic functions.</p>
</blockquote>
<p>I guess that answers my question :)</p>



<a name="242523806"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523806" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523806">(Jun 13 2021 at 18:24)</a>:</h4>
<p>Thanks for the useful information! I think I'm going to look into it and see if I can reduce the size a bit :)</p>



<a name="242523811"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523811" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523811">(Jun 13 2021 at 18:24)</a>:</h4>
<p><span class="user-mention silent" data-user-id="307537">Noah Lev</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523750">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523737">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="307537">Noah Lev</span> for generic functions, they are monomorphized in downstream crates</p>
</blockquote>
<p>Oh, of course! So do monomorphic functions just have object code and polymorphic functions just have MIR, or do monomorphic functions have MIR too?</p>
</blockquote>
<p>All functions have MIR but unless <code>-Zalways-encode-mir</code> is used, it is omitted from the crate metadata when not necessary. <code>-Zalways-encode-mir</code> is for example used by miri as it needs MIR for all functions to interpret.</p>



<a name="242523905"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523905" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523905">(Jun 13 2021 at 18:26)</a>:</h4>
<blockquote>
<p>hygiene bytes: 3675259</p>
</blockquote>
<p>That is a lot of bytes. I think it mostly shouldn't be necessary except for decl macro definitions and maybe nicer diagnostics.</p>



<a name="242523952"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242523952" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242523952">(Jun 13 2021 at 18:27)</a>:</h4>
<p><span class="user-mention silent" data-user-id="133247">bjorn3</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523811">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="307537">Noah Lev</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523750">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523737">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="307537">Noah Lev</span> for generic functions, they are monomorphized in downstream crates</p>
</blockquote>
<p>Oh, of course! So do monomorphic functions just have object code and polymorphic functions just have MIR, or do monomorphic functions have MIR too?</p>
</blockquote>
<p>All functions have MIR but unless <code>-Zalways-encode-mir</code> is used, it is omitted from the crate metadata when not necessary. <code>-Zalways-encode-mir</code> is for example used by miri as it needs MIR for all functions to interpret.</p>
</blockquote>
<p>Thanks! Yeah, that's what I meant by "have"; I should have been clearer by saying "have <em>in metadata</em>" :)</p>



<a name="242524384"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242524384" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242524384">(Jun 13 2021 at 18:36)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/131828-t-compiler/topic/metadata.20size/near/242523676">said</a>:</p>
<blockquote>
<p>What are they for?</p>
</blockquote>
<p>This is the code responsible for collecting the stats:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="w">            </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">zero_bytes</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span>
<span class="w">            </span><span class="k">for</span><span class="w"> </span><span class="n">e</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">opaque</span><span class="p">.</span><span class="n">data</span><span class="p">.</span><span class="n">iter</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">                </span><span class="k">if</span><span class="w"> </span><span class="o">*</span><span class="n">e</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">                    </span><span class="n">zero_bytes</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="mi">1</span><span class="p">;</span><span class="w"></span>
<span class="w">                </span><span class="p">}</span><span class="w"></span>
<span class="w">            </span><span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>It seems like any byte in the metadata that is zero is counted? So it's not necessarily something like padding.</p>



<a name="242524387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242524387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242524387">(Jun 13 2021 at 18:36)</a>:</h4>
<p>I wonder why it's reported, though.</p>



<a name="242525852"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242525852" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jubilee <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242525852">(Jun 13 2021 at 19:10)</a>:</h4>
<p>that's still like 40% of libstd tho' lol.</p>



<a name="242526914"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242526914" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242526914">(Jun 13 2021 at 19:36)</a>:</h4>
<p>Integers are already encoded as LEB128 to save space, so I am not sure what those zeros are used for.</p>



<a name="242526941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242526941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242526941">(Jun 13 2021 at 19:36)</a>:</h4>
<p>Maybe it is like the <code>LOCAL_CRATE</code> <code>CrateNum</code> or <code>DUMMY_SPAN</code>?</p>



<a name="242539340"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242539340" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242539340">(Jun 14 2021 at 00:33)</a>:</h4>
<p><code>Span</code> seems to use LEB128 for <code>lo</code> and <code>hi</code> FWIW.</p>



<a name="242561401"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/metadata%20size/near/242561401" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/metadata.20size.html#242561401">(Jun 14 2021 at 07:32)</a>:</h4>
<p>For <code>DUMMY_SP</code> both are zero and the <code>SyntaxContext</code> is also zero.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>